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- The MAILING DATE of this communication appears on the cover sheet with the correspondence address 
Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1.136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- if the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 1 33). 

- Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

1 )^ Responsive to communication(s) filed on 23 March 2004 . 
2a)Kl This action is FINAL. 2b)d This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quay/e, 1935 CD. 11.453 0.G. 213. 
Disposition of Claims 

4) E3 Claim(s) 1-23 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) IE Claim(s) 4,5,11 and 12 is/are allowed. 

6) |E Clairn(s) 1-3,6-10 and 13-23 is/are rejected. 

7) Q Claim(s) is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) Kl The specification is objected to by the Examiner. 

10) D The drawing(s) filed on is/are: a)D accepted or b)Q objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1 .85(a). 

11) ^3 The proposed drawing correction filed on 23 March 2004 is: a)E3 approved b)D disapproved by the Examiner. 

If approved, corrected drawings are required in reply to this Office action. 

12) D The oath or declaration is objected to by the Examiner. 
Priority under 35 U.S.C. §§ 1 1 9 and 1 20 

13) ^ Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 19(a)-(d) or (f). 

a)|ElAII b)D Some*c)D None of: 

1 .[3 Certified copies of the priority documents have been received. 

2. \Z\ Certified copies of the priority documents have been received in Application No. . 

3. D Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 

14) D Acknowledgment is made of a claim for domestic priority under 35 U.S.C. § 119(e) (to a provisional application). 

a) □ The translation of the foreign language provisional application has been received. 

15) D Acknowledgment is made of a claim for domestic priority under 35 U.S.C. §§ 120 and/or 121. 
Attach ment(s) 

1 ) Notice of References Cited (PTO-892) 4) □ Interview Summary (PTO-413) Paper No(s). 
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DETAILED ACTION 

1 . The text of those sections of Title 35, U.S. Code not included in this action can 
be found in a prior Office action. 

Response to Amendment 

2. The response filed 23 March 2004 was filed to the following effect: 

- The title was changed as indicated and the objection is withdrawn as satisfied. 

- The specification was changed as indicated and the objection is withdrawn. 

- The claims were changed as indicated and the objections are withdrawn as satisfied. 

Response to Arguments 

3. Applicant's arguments filed 23 March 2004 have been fully considered but they 
are not persuasive. 

- Regarding claims 1 and 8, the HMM of the cited prior art is used as claimed, as 
indicated in the Abstract of Komori et ai which teaches that "1 st output probabilities 
are formed for inputted speech using the small number of mixtures type HMM and 
2 nd output probabilities are formed for the input speech using the large number of 
mixtures type HMM for selected states corresponding to the highest output 
probabilities of the 1 st type HMM. 
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- The added component, which details the activity of the segment is, in fact, being 
registered has been addressed explicitly as a feature of the claim. 

Drawings 

4. The replacement drawing (figure 6) was received on 23 March 2003. 
The drawing is accepted. 

Claim Rejections - 35 USC § 103 

5. This application currently names joint inventors. In considering patentability of 
the claims under 35 U.S.C. 103(a), the examiner presumes that the subject matter of 
the various claims was commonly owned at the time any inventions covered therein 
were made absent any evidence to the contrary. Applicant is advised of the obligation 
under 37 CFR 1 .56 to point out the inventor and invention dates of each claim that was 
not commonly owned at the time a later invention was made in order for the examiner to 
consider the applicability of 35 U.S.C. 103(c) and potential 35 U.S.C. 102(e), (f) or (g) 
prior art under 35 U.S.C. 1 03(a). 

Maddox, Komori et al' 975 & ' 396 & Chu 

6. Claims 1, 6, 8, 13, 15, 17, 18, 20, 21 and 23 are rejected under 35 U.S.C. 103(a) 
as being unpatentable over Maddox (U.S. Patent 6,000,024 A) in view of Komori et al' 975 
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(U.S. Patent 5,812,975) and further in view of Komori et al* 96 (U.S. Patent 5,787,396) 
and further in view of Chu (U.S. Patent 6,374,210 B1). 



7. Regarding claims 1 and 8, the invention for a parallel computing system by 
Maddox reads on the feature of the claim for a speech signal processing apparatus as 
follows: 

- Maddox (column 6 lines 44-46) reads on the feature of a speech segment search 
means for searching a speech database for speech segments that satisfy a phonetic 
environment; 

- Maddox does not mention HMM. The state transition model design and voice 
recognition of Komori et af 975 (column 3 lines 32-38) reads on the feature of HMM 
learning means for computing HMMs of phonemes on the basis of a search result of 
that speech segment search means and (column 6 lines 52-53) reads on the feature 
of segment recognition means for performing segment recognition of the speech 
segments on the basis of the HMMs of the phonemes; 

- Maddox does not mention registering segments. The speech recognition method of 
Komori et al 1396 (column 4 lines 24-30) reads on the feature of registration segment 
determination means for determining a speech segment to be registered in a 
segment dictionary in accordance with a segment recognition result of that segment 
recognition means. 

- It would have been obvious to a person of ordinary skill in the art of speech signal 
processing at the time of the invention to apply the method/teachings of Komori et al 
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to the device/method of Maddox so that differences in phone environments - such 
as preceding and succeeding phones - can be classified finely. 

With regard to the additional feature of the claim that registration segmentation means 
for determining (selecting) a speech segment used in the computation of the HMMs by 
the HMM learning means and registering the speech segment in a segment dictionary in 
accordance with (the) segment recognition result (of the segment recognition means), 
Komori et al indicates that HMM makes a contribution to registering a speech segment 
in a segment dictionary (105->108 in figure 1 , 204->208 in figure 2, etc.) without 
disclosing the mechanism depicted. Chu, with the invention fot the automatic 
segmentation of a text, reads on this further feature of registering the speech segment 
in a segment dictionary With the update function (140 in figure 1 - see column 7 lines 
30-33). It would have been obvious to a person of ordinary skill in the art of speech 
signal processing at the time of the invention to apply the method/teachings of Chu to 
the device/method of Maddox & Komori et al to incorporate new words/segments into 
the segment lexicon (dictionary). 

8. Regarding claims 6 and 13 as understood by the Examiner; the claims are set 
forth with the same limits as claims 1 and 8, respectively. Maddox does not mention 
registering segments. Komori et al 1396 (2032 in figure 8) reads on the feature of speech 
segments having likelihood not less than a predetermined value registered in the 
segment dictionary which would have made it obvious to a person of ordinary skill in the 
art of speech signal processing at the time of the invention to apply the method and/or 
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teachings of Komori et al 3396 to the device/method of Maddox so as to have recognition 
results reflect an acceptable comparison. 

9. Regarding claim 15, the claim is set forth with the same limits as claim 8. 
Maddox (claim 7 line 32) reads on the feature of a computer readable storage medium 
storing a program for implementing a method cited. 

1 0. Regarding claims 1 7 and 20, the features of the claims are the same as those 
found in claims 1 and 8 and the claims are rejected for the same reasons. 

11. Regarding claims 18 and 21; the claims are set forth with the same limits as 
claims 17 and 20, respectively. Maddox is silent as to use of HMM. Komori et af 396 
(column 4 lines 1 5-31 ) reads on the feature that obtains a maximum likelihood HMM 
which has a maximum likelihood with one of the plurality of speech segments from the 
HMMs corresponding to the phonemes, checks if the one speech segment is a speech 
segment used in learning of the maximum likelihood HMM, and selects the one speech 
segment when the one speech segment is a speech segment used in learning of the 
maximum likelihood HMM. It would have been obvious to a person of ordinary skill in 
the art of speech signal processing at the time of the invention to apply the method 
and/or teachings of Komori et al to the device/method of Maddox so as to process faster 
without reducing recognition accuracy. 
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12. Regarding claim 23, the claim is set forth with the same limits as claim 20. 
Maddox (claim 7 line 32) reads on the feature of a computer readable storage medium 
storing a program for implementing a method cited. 

Maddox, Komori et al 975& ' 396 , Chu & Rosenberg 

13. Claims 2 and 9 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Maddox in view of Komori et al 975 and further in view of Komori et al' 396 and further in 
view of Chu and further in view of Rosenberg ("Connected Sentence Recognition Using 
Diphone-Like Templates" International Conference on Acoustics, Speech, and Signal 
Processing, April 1988). 

14. Regarding claims 2 and 9; the claims are set forth with the same limits as claims 
1 and 8, respectively. Maddox is silent on the feature of diphones or biphones. 
Rosenberg (lines 7-9 left column page 473) reads on the feature that segment 
recognition means adopts diphones as units of the phonemes, categorizes speech 
segments into four categories CC, CV, VC, and VV (C: a consonant, V: a vowel), and 
performs segment recognition in each category (in the end of that paragraph). It would 
have been obvious to a person of ordinary skill in the art of speech signal processing at 
the time of the invention to apply the method/teachings of Rosenberg to the device 
and/or method of Maddox , Komori et al 975 & 39 & Chu because defining units larger than 
phones will be easier to work with, segment and label by virtue of containing greater 
variations and context effects. 
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Maddox, Komori et a i' 975& ' 396 n Chu & Tominaga 

15. Claims 3 and 10 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Maddox in view of Komori et al 975 and further in view of Komori et al' 396 and further in 
view of Chu and further in view of Tominaga (U.S. Patent 5,31 1 ,429 A). 

16. Regarding claims 3 and 10; the claims are set forth with the same limits as 
claims 1 and 8, respectively. Maddox does not mention registering segments. 

The speech recognition method of Komori et al 396 (column 4 lines 24-30) reads 
on the feature of determining a speech segment to be registered in a segment 
dictionary in accordance with a segment recognition result of that segment recognition. 

The maintenance support method and apparatus for natural language processing 
system of Tominaga (column 1 1 lines 19-36) reads on the feature that checks if a 
speech segment pattern which matches a speech segment that is not successfully 
recognized by that segment recognition means, and registers that speech segment in 
the segment dictionary if the corresponding speech segment pattern is found. 

It would have been obvious to a person of ordinary skill in the art of speech 
signal processing at the time of the invention to apply the method/teachings of Komori 
et al & Tominaga to the device/method of Maddox , Komori et al 975 & 39 & Chu to avoid 
duplicate entries. 
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Maddox, Komori et al' 975& ' 39 \ Chu & Richardson etal 

17. Claims 7 and 14 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Maddox in view of Komori et al 975 and further in view of Komori et al' 396 and further in 
view of Chu and further in view of Richardson et al (U.S. Patent 5,926,784 A). 

18. Regarding claims 7 and 14; the claims are set forth with the same limits as 
claims 6 and 13, respectively. Maddox does not mention registering segments. 

In natural language parsing using Podding, Richardson et al (column 1 lines 21- 
25) reads on the feature that registers, in the segment dictionary, speech segments 
having upper values (claims 39 and 41 ) obtained by normalizing the likelihood by 
durations of the speech segments or likelihood having the values not less than a 
predetermined value. 

It would have been obvious to a person of ordinary skill in the art of speech 
signal processing at the time of the invention to apply the method/teachings of 
Richardson et al to the device/method of Maddox , Komori et al 975 & 39 & Chu to improve 
the efficiency by assigning probabilities to syntax rules. 

Maddox. Komori et al' 975 & ' 396 < Chu, Tominaga, Fukada etal& Huang etal 193 

19. Claim 16 is rejected under 35 U.S.C. 103(a) as being unpatentable over Maddox 
in view of Komori et al 975 and further in view of Komori et al 396 and further in view of 
Chu and further in view of Tominaga and further in view of Fukada et al (U.S. Patent 
5,845,047 A) and further in view of Huang et al 193 (U.S. Patent 5,913,193 A). 
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20. Regarding claim 16, the claim is set forth with the same limits as claim 8. The 
feature of a segment dictionary in which speech segments are registered is cited in 
claim 8 and the rejections are applied to this claim for the same reasons. 

Where Maddox is silent on the subject of language analysis, Tominaqa (column 3 
lines 46-56) reads on the feature for performing language analysis of input text data. 

Where Maddox is silent on the subject of prosody, Fukada et al (column 6 line 
59) reads on the feature of generating prosody on the basis of an analysis result of that 
language analysis means and the runtime acoustic unit selection for speech synthesis 
of Huang et al 193 (column 3 lines 1-4) reads on the feature that search that segment 
dictionary on the basis of the prosody generated by that prosody generation means to 
select corresponding speech segments and, with (column 1 line 66) reads on modifying 
and concatenating the speech segments selected by that speech segment selection 
means and (1 32 in figure 5) for reproducing speech on the basis of the result modified 
by that speech segment modification/concatenation means. 

It would have been obvious to a person of ordinary skill in the art of speech 
signal processing at the time of the invention to apply the method/teachings of 
Tominaqa, Fukada et al & Huang et al 193 to the device/method of Maddox , Komori et 
al 975 & 39 & Chu so as to improve the quality of synthetic speech by considering 
additional attributes of language. 
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Maddox, Komori et al 975& ' 396 . Chu & Huang etal' 193 

21. Claims 19 and 22 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Maddox in view of Komori et al 975 and further in view of Komori et al' 396 and further 
in view of Chu and further in view of Huang et al 193 (U.S. Patent 5,913,193 A). 

22. Regarding claims 19 and 22; the claims are set forth with the same limits as 
claims 17 and 20, respectively. Maddox is silent as to speech synthesis. Huang et 
al 193 (36 in figure 1 ) reads on the feature of producing synthetic speech using the 
segment dictionary (22->->36 in figure 1) ... which would have made it obvious to a 
person of ordinary skill in the art of speech signal processing at the time of the invention 
to apply the method/teachings of Huang et al to the device/method of Maddox , Komori 
et al' 975 & 39 & Chu so as to generate natural-sounding speech. 

Allowable Subject Matter 

23. Claims 4-5 and 11-12 are allowed. 

24. The following is a statement of reasons for the indication of allowable subject 
matter: 

- The present invention is directed to building and maintaining speech segment 
dictionaries. 
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- Claims 4 and 1 1 identify the uniquely distinct feature that "registers a speech 
segment in the segment dictionary when the number of speech segments 
recognized is not less than a predetermined value " 

The closest prior art, Yokota et a/ , discloses registering recognized speech 
segments but fails to anticipate or render the above underlined limitations obvious. 

- Claims 5 and 1 2 depend on claims that were found to be allowable and so would 
they be allowed as a consequence. 

25. As allowable subject matter has been indicated, applicant's reply must either 
comply with all formal requirements or specifically traverse each requirement not 
complied with. See 37 CFR 1.111(b) and MPEP § 707.07(a). 

Conclusion 

26. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. 

- Goronzv et al (U.S. Patent Publication 2002/0095282 A1 )method for online 
adaptation of pronunciation dictionaries. 

- Shibata (U.S. patent 5,678,054 A) data searching device. 



27. Applicant's amendment necessitated the new ground(s) of rejection presented in 
this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP 
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§ 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 
CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the date of this final action. 

28. Any inquiry concerning this communication or earlier communications from the 
Examiner should be directed to Daniel A. Nolan at telephone (703) 305-1368 whose 
normal business hours are Mon, Tue, Thu & Fri, from 7 AM to 5 PM. 

If attempts to contact the examiner by telephone are unsuccessful, supervisor 
Richemond Dorvil can be reached at (703)305-9645. 

The fax phone number for Technology Center 2600 is (703)872-9314. Label 
informal and draft communications as "DRAFT" or "PROPOSED", & designate formal 
communications as "EXPEDITED PROCEDURE". Formal response to this action may 
be faxed according to the above instructions, 

or mailed to: Mail Stop AF (or CPA, etc. - see Official Gazette, 04 November 2003) 

P.O. Box 1450 
Alexandria, VA 22313-1450 
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or hand-deliver to: Crystal Park 2, 

2121 Crystal Drive, Arlington, VA, 
Sixth Floor (Receptionist). 

Any inquiry of a general nature or relating to the status of this application or 
proceeding should be directed to Technology Center 2600 Customer Service Office at 
telephone number (703) 306-0377. 



Daniel A. Nolan 
Examiner 
Art Unit 2654 



DAN/d 

April 8, 2004 




